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Abstract  i._  ..  _ 

Average  information  capacity  is  determined  for  a  class  of  communication 
channels  containing  additive  noise.  Gaussian  noise  processes  and  a  large 
class  of  nonGaussian  processes  are  included.  The  constraint  on  the  trans¬ 
mitted  signals  is  given  in  terms  of  an  increasing  family  of  finite-dimensional 
subspaces.  The  results  apply  to  the  classical  discrete-time  channel  and  to 
continuous-time  channels  with  fixed  signal  duration. 
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Introduction 

Coding  capacity  of  additive  Gaussian  channels  is  a  fundamental  problem  in 
the  Shannon  information  theory  [1],  [4].  However,  concrete  results  on  this 
problem  have  actually  been  obtained  only  for  special  cases  involving 
stationary  noise. 

In  this  paper,  capacity  is  obtained  for  dimension-limited  Gaussian 
channels.  The  result  also  holds  for  a  large  and  important  (in  applications) 
class  of  nonGaussian  channels.  Bounds  on  capacity  are  obtained  for  a  larger 
class  of  nonGaussian  channels.  The  results  given  are  for  information 
capacity,  but  in  a  form  suitable  to  use  for  obtaining  coding  capacity;  that 
application  is  given  elsewhere  [3]. 

It  will  be  seen  that  the  development  entails  extensive  use  of  the 
spectral  representation  of  a  self-adjoint  linear  operator.  This  operator 
defines  the  relation  between  the  energy  constraint  on  transmitted  signals  and 
the  channel  noise  covariance.  In  particular,  the  essential  spectrum  of  this 
operator  plays  (fittingly!)  an  essential  role  in  the  development. 


Problem  Formulation 

The  channel  is  described  by  Y  =  X  +  N,  where  Y,  X,  and  N  represent 
measurable  stochastic  processes  in  an  underlying  probability  space  (0,0. u), 
with  paths  a.s.  in  a  real  separable  Hilbert  space  H.  H  has  inner  product 


<*,•>  and  norm  11*11.  N  is  represented  by  a  probability  p^;  p^  need  not  be  For 

I 

countably  additive.  Thus,  p^  is  defined  on  the  cylinder  sets  of  H. 

i 

Associated  with  ^  Is  a  covariance  operator  that  is  linear,  bounded,  self-  on _ _ 


adjoint,  and  non-negative: 


<R^u,v>  =  Jpj<x,u><x,  v>dp^(x) 


By - 

Distribution/ 

jPN  Availability  Codas 
)  Avail  and/or 

w\  /  Dlst  Special 
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We  assume  WLOG  that  p^  is  strictly  positive  with  zero  mean. 

In  order  to  obtain  finite  capacity,  any  set  of  constraints  on  the  signal 
X  must  involve  (explicitly  or  implicitly)  a  self-adjoint  bounded  linear 
operator  R^  which  is  strictly  positive  and  satisfies 

In  this  representation  (I+S)  *  exists  and  is  bounded,  but  the  self-adjoint 

operator  S  need  not  be  bounded.  See  [2]  for  details. 

In  order  to  formulate  the  problem  in  a  form  consistent  for  application  to 

coding  capacity,  the  constraints  will  involve  an  increasing  family  of  linear 

manifolds  (H  ) .  H  C  H  ,  C  H,  dim[H  ]  =  n.  P  will  be  the  projection 
v  n7  n  n+1  L  nJ  n 

operator  in  H  with  range  equal  to  H  .  will  be  the  projection  operator 

n 

% 

with  range  equal  to  rangefR^P^]. 

For  each  n.  the  constraint  on  the  class  of  admissible  transmitted  signals 

x  is  given  as  follows: 

u  Pnx  C  Hn  a.e. 

2)  E  IIP  xll?  $  nP 

'  p^  n  W.n 

where  p^  is  the  cylindrical  probability  describing  the  transmitted  signal. 

2  2 

P  <  ®  is  fixed,  and  llyll,„  =  Hull  .  where  u  is  the  unique  element  in  H 

W.n  n 

satisfying  (P^R^P^  “u  =  y.  For  each  n.  this  defines  a  channel  of  the  form 
Yn  =  Xn  +  Nn,  where  Xn  represents  a  stochastic  process  with  paths  a.s.  in 

and  described  by  a  (zero-mean)  probability  p^,  p^(A)  =  p^°Pn*[A]  for  each 

7i  n  ^  j 

Borel  set  in  H^.  N  is  defined  by  the  zero  mean  probability  p^  =  M^°Pn  • 

The  constraints  (1)  and  (2)  are  equivalent  to  the  following: 

l'>  ^  =  1 

2*)  En"xNW.nlnP 
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where  p!^  is  a  probability.  The  equivalence  of  these  two  definitions  follows 

from  the  fact  that  a  cylindrical  probability  on  a  Hilbert  space  is  uniquely 

defined  by  its  projections  on  the  f ini te-dimensional  subspaces. 

For  any  n  £  1,  let  be  the  zero-mean  Gaussian  probability  on  H  with 

tin  n 

n  ~  ~ 

covariance  operator  R^  =  P  R^P  .  <^eve^°Pment  here  will  make  use  of  the 

relative  entropy  H^(N).  n  £  1, 


,  n 

H^(N)  =  ;H  (log  >-]< 
n  dp_MJ 


GN 

We  define  H^(N)  =  00  if  dpjj/dp^  does  not  exist.  The  most  precise  results  (an 

exact  value  of  the  capacity)  will  be  obtained  for  channels  such  that 

TS  i  1^(N)  =  0.  Of  course,  this  includes  all  channels  defined  by  Gaussian 
n 

cylinder  set  measures:  it  also  includes  many  nonGaussian  channels.  In  fact, 

let  H^(N)  =  sup  H^,(N).  Many  important  nonGaussian  channels  satisfy 
n 

H^N)  <  For  example,  if  ^  s  py#p^,  where  Py  is  zero-mean  Gaussian  with 
covariance  operator  Ry  and  p^  is  zero  mean  with  covariance  operator  =  R^TR^ 
for  T  trace-class,  then  H^(N)  <  «  regardless  of  the  other  characteristics  of 
M*  [3]. 

Two  concrete  examples  of  this  setup  are  the  classical  discrete-time 

channel  and  the  continuous- time  channel  with  fixed  (finite)  transmission  time 

for  the  code  words.  In  the  classical  discrete-time  channel.  H  =  £n.  H  =  (x 

Z  n 

in  =  0,  i  >  n).  and  R^  is  the  noise  covariance  matrix.  R^  is  also 

defined  by  a  covariance  matrix;  when  the  channel  noise  is  stationary,  then  R^ 
is  typically  defined  by  a  time-invariant  linear  filter  h  with  Fourier 
transform  h.  |h|  then  gives  a  spectral  density  which  defines  the  covariance 
operator  R^.  The  present  paper  is  limited  to  obtaining  expressions  for  the 
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average  information  capacity,  and  related  results,  in  the  above  framework.  In 
[3],  these  results  are  applied  to  the  classical  discrete-time  channel  to 
obtain  new  results  on  coding  capacity.  Capacity  of  the  stationary  channel 
when  =  I  has  been  known/assumed  for  many  years;  however,  it  is  apparently 
only  recently  that  a  rigorous  proof  has  been  given  for  the  channel  with  memory 
[5]. 

The  other  principal  concrete  example  is  for  H  =  L^CO.T],  T  <  °°,  with  H  = 
span{e^ , . . . , en) ,  where  {e^.  k  >  1}  is  an  infinite  o.n.  set.  The  application 
to  such  channels  is  described  in  [3],  which  also  contains  a  number  of  specific 
resul ts . 

For  each  n,  one  thus  has  a  channel  in  H  ,  described  by  the  joint 

n  j  j 

probability  p^y(A)  =  (x.y)  :  x  +  y  e  M  for  A  a  Borel  set  in  H^xH^. 

The  mutual  information  is  then 


and  the  sup  is  over  all  Borel-measurable  partitions  A.,  A_ . A,  of  H  xH  . 

1  2  k  n  n 

The  capacity  is  C^(nP)  =  sup  I(p^y),  where  the  sup  is  over  all  p^  such  that 

E  llxll^  <  P. 
n  W,n  ~ 


The  channel’s  average  information  capac 


ity  is  then  C^(P)  -  lim  ^C^(nP). 


n-*» 


Of  course,  coding  capacity  is  the  important  operational  parameter  of  the 

channel.  If  p^  is  Gaussian,  then  it  can  be  shown  [3]  that  coding  capacity  is 

00 

equal  to  the  average  information  capacity  C^( P )  for  the  class  of  channels 

N 

considered  here.  In  addition,  if  p^  is  not  Gaussian,  but  satisfies  <  «\ 

CD 

then  C^(P)  gives  an  upper  bound  on  the  coding  capacity  [3]. 
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CO 

The  program  here  is  to  give  a  general  expression  for  C^(P),  valid  for  all 

channels  satisfying  the  above  model.  Such  channels  are  described  by  a  noise 

covariance  operator  R^,  a  constraint  operator  R^,  an  increasing  family  of 

finite-dimensional  linear  manifolds  (Hn),  and  an  "average  signal-to-noise 

energy"  limitation  P.  Equivalently,  the  channel  is  characterized  by  P,  by  S 

(the  operator  defining  the  relation  between  the  noise  covariance  and  the 

channel  covariance) ,  and  by  (P^  ),  a  monotone  increasing  family  of  projection 

n 

operators,  P^  having  range  equal  to  range(R^P^). 
n 

As  part  of  the  development,  it  will  be  seen  that  the  results  depend  on 
the  spectrum  cr(S)  of  S,  but  only  upon  the  part  belonging  to  the  essential 
spectrum  CTess(S).  This  set,  which  is  also  the  "limit  points  of  the  spectrum," 
consists  of  limit  points  of  distinct  eigenvalues,  eigenvalues  of  infinite 
multiplicity,  and  points  of  the  continuous  spectrum  [7]. 


Preliminary  Results 

The  average  capacity  C^(P)  is  defined  as  lim  C^(nP) .  C^(nP),  the 

n-*» 

information  capacity  of  the  channel,  is  given  by  the  following  well-known 
result . 


Lemma  1  [2,5,6], 


N(n) 

'A  2  log 


i =1 


B(n)+1 


N(n) 

<  cJ(nP)  <  A  2  log 

i =1 


B(n)+l 


1  P 


where  0?  <  /£  i  ...  <  are  the  eigenvalues  of  S  ,  N(n)  -  sup{i 
x  z  n  n 

p1^  <  B(n)},  and  B(n)  is  defined  by 


* 

<  H.  • 


1  N(n)  n 

P-i  2  (B(n,  -  if). 

1=1 


The  next  lemma  gives  several  fundamental  relations. 
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Lemma  2:  Let  S:  H  -»  H  satisfy  =  R^(I+S')R^,  define  ^  =  P^R^P^  and 

R„  =  P  R„P  ,  and  let  S  :  H  -»  H  satisfy  R„  -  RS  (I  +S  )R^  ,  for 

IV, n  n  If  n  n  n  n  a  N.n.  w,nv  n  n'  W,r 

n  >  1,  uihere  I  is  the  identity  in  H  .  Then: 

n  M  n 

(1)  R^  ^  =  P^R^V^,  inhere  V^:  H  ^  is  a  partial  isometry  with  initial 


set  equal  to  range(R^P^)  and  final  set  H 

(2)  S  =  V  SV*,  n  >  1. 

(3)  Let  be  the  projection  operator  with  range  equal  to  = 

range(R^P^)  .  The  eigenvalues  of  P^SP^  (and  their  mul  tiplici  ty) 

are  the  same  as  those  of  S  . 

n 

Proof .  (1)  follows  from  results  in  £4].  (2)  is  obtained  by  equating  the  two 

definitions  of  R^  n<  yielding  PnR^( I+SJR^P^  =  n(^n+®n)^-  then  applying 

part  (1).  (3)  follows  from  the  fact  that  range(Pw  )  =  H™  =  range(V  ). 

Wxi  wn  n  u 

The  family  of  partial  isometries  (Vn)  plays  a  key  role  in  the  analysis  of 
capacity.  Consider  II  =  2^  and  the  classical  discrete-time  channel. 

Range(R^Pn),  the  range  space  of  V  ,  is  then  equal  to  the  span  of  the  first  n 

14  *  ~ 

columns  of  R^.  If  R^  =  I,  this  of  course  gives  =  P^,  the  projection 

onto  =  {x:  =  0,  i  >  n}.  This  is  the  simplest  possible  case,  and  enables 

one  to  immediately  give  the  capacity  of  the  stationary  Gaussian  channel  as  a 

corollary  to  the  general  expression  for  the  capacity  [3]. 

{e. ,  i  >  1}  will  be  used  to  denote  any  o.n.  set  in  H  such  that  H  = 

span{e ^ , . . . , e^} .  Similarly,  {u ^ ,  i  >  1}  will  denote  any  o.n.  set  such  that 

H...  =  span{u, . u  }.  The  partial  isometry  V  is  defined  by  V  u.  =  e., 

i  <  n.  For  fixed  n,  {v?:  i  <  n}  will  denote  o.n.  eigenvectors  of  Pw  SPW 


W  ~‘W 
n  n 


corresponding  to  the  eigenvalues  (P?}:  P™  SPW  v°  =  |3nvn  for  i  <  n 

1  nr  W  1  11 


Info. Cap.  of  DLC  -  9/5/89  -  6 


Let  {E^,  X  €  IR}  be  the  left-continuous  resolution  of  the  identity  [7]  for 

00 

S,  so  that  for  x  in  the  domain  2>(S)  of  S,  Sx  =  J_^XdE^x.  For  S  bounded,  this 
representation  is  the  limit  of  a  finite-limit-point  approximation  (FLPA)  to  S, 
(S^),  defined  as  follows.  For  S  bounded,  1  <  i  <  K,  let  -1  =  a!5  <  < 


and 


IIS* 


<  a^,  where  l+aj^_j  i  III+SII  <  1+aj^.  Let  be  the  range  of  E  ^  -  E  ^ 

a .  a . 

l  l-l 

set  Ev.  =  Ej.-Elr  .  Let  S^  =  2  0^E  v,  where  0^  €  [a^  .  ,a^).  Then 

.K  K.  K.  .  -  1  .K,  1  I”*!  1 


AV 

l 


a.  a.  , 
l  l-l 


i=l  *  A. 
i 


Sll  -»  0  as  K  “  and  sup{a^  -  a^_^:  *  <  K}  -*  0  [7].  If  S  is  not  bounded, 

M 


then  for  any  x  in  2)( S),  Sx  =  lim  2  /^^AdE^x,  where  for  each  n,  Jn  ,AdE7 

M-*=°  n=0  n  n 

exists  as  the  limit  of  a  FLPA  (S^[n] )  and  lU^AcLE^  -  SK[n]ll  -+  0  as  K  -»  ®  [7], 

k 

The  operators  (T^) ,  T^  =  J^^XdE^,  are  pairwise  orthogonal:  T^:  H  ->  H^. 


1  if  j  ^  k.  Also.  T^  =  ^AdE^,  where  is  the  restriction  of 

H^.  Thus.  H  =  ®H^,  E^:  ri  -»  1^,  Sx  =  T^x  whenever  x  €  95(S)  D  H^. 
k 

If  S  is  not  bounded,  then  there  exists  a  FLPA  to  S  on  [-1.B]  for  any 

M  k 

finite  B  >  1.  That  is,  for  any  x  in  S(S) ,  Sx  =  lim  2  /.  .XdELx,  where  for 

Mh»  k=0 

each  k,  /j^AdE^  *s  t*ie  *n  the  uniform  operator  topology  of  a  FLPA. 

Thus,  if  B  and  B'  are  finite,  then  the  bounded  operators  EgS  and  Eg.S  each 

have  a  FLPA,  and  the  approximating  sequences  are  consistent:  if  B‘  >  B,  then 

any  FLPA  (A^)  to  Eg.S  provides  a  FLPA  to  EgS,  given  by  (EgA^). 

If  (SK)  is  a  FLPA  to  (S).  then  mJJ(  j)  will  denote  the  number  of 

eigenvalues  (repeated  according  to  their  multiplicity)  of  P^n(S  )P^n  c^at 
K  K 

in  the  interval  [a.  ,,a.). 

L  j-i  r 

The  integral  representation  of  S  will  play  a  key  role  in  the  following 

analysis.  For  S  bounded,  III+SII  <  1+0,  and  f  any  continuous  function  on 

0+ 

(0,  1+0],  it  is  noted  that  [f(I+S)]x  =  J-  f(l+A)dE^x  for  all  x  in  H  [7], 


to 
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K 

Note  that  for  any  fixed  n,  the  ordered  eigenvalues  of  (P^S  P  )  converge 

to  those  of  P  SP  . 

n  n 

Lemma  3 .  Let  B  be  any  positive  real  number,  with  f  any  continuous 

function  on  [0,B],  For  all  n  >  1,  define 
n  K 

where  K  >  1  and  (I+sP)  is  a  FLPA  to  I+S.  Then,  A n=  0  for  all  n  >  1, 
and  the  convergence  is  uniform  for  all  n  >  1 . 

Proof .  First,  assume  that  S  is  bounded  and  that  B  >  III+SII.  Let  f(x)  =  xP  foi 
any  integer  p  >  0  and  suppose  that  1+0^  i  M  <  00 ,  all  K  >  1  Then, 


A  =  - 
n  n 


2  (1+P")P  -  lim  2  (H-oW^j) 
i=l  K h®  j=l  J 


It  is  obvious  that  =  0  if  p  =  0;  thus,  suppose  p  >  1. 


A  =  - 
n  n 


lim  2  (l+pK’n)P  -  lim  2  ( 1+0K)PMK( j) 

K-hoo  i=i  1  K-*°  j=l  J  n 

where  (/3^’n)  are  eigenvalues  of  V  S^V 


n  n 


K  r 
lim  2 
K  j=l 


2  (i+/JK-n)p  _ 

i:^n€[aK  t.aK)  J 

i  L  .1-1  j' 


1  K 
<  -  1  im  2 


2  (l+^'n)P  -  (lWhPM*J(j) 

K  j=l  . . oK.n  r  K  K,  J 

l -p .  €[a  .  . ,a  .) 

i  L  J-l  J 


1  K 
<  —  1  im  2 
~  n 


2  (lWWj-a*  )P  -  ( 1+0^)PM^(  j ) 

K  J*lLi:^.n€[aK  K  J  J  J 

i  L  J-l  j' 


1 


K 


<  -  lim  2  Nfrj) 
n  K  j=l  n 


(l+0K+aK-aK  ,)P  -  (1+0*)P 
J  J  J-l  K' 
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K  r 

<  ^  lim  2  M*(j)  (l+0^eK)P  -  (1 +0*)1 
K  j=l 


where  =  sup{a^-a^  . .  j  <  K}. 
K  J  J-l 


Sl&J)  =  n.  every  K  >  1.  Thus, 


A  <  lim  ( l+0^+ev)P  -  (1+0{S)P  <  lim|(M+t„)P  -  Mp  -  0. 


since  the  sequence  (0^)  is  bounded  and  -»  0.  The  same  procedure  obviously 
gives  =  0  when  f  is  any  polynomial.  Note  that  the  convergence  is  uniform 
for  all  n  >  1 . 

Next, suppose  that  f  is  any  continuous  function  on  [0,M],  take  e  >  0.  and 

let  Q  =  Q(e.f)  be  a  polynomial  such  that  sup  | Q ( X )  -  f ( X )  |  <  e.  Then 

0<A<M 

n  K  K 

A  <  2  Q(l+0?)  -  lim  2  Q(l+0V(j)  +  ^  ne  +  e  lim  2  M*5( j) 

n  i=l  Knm  j=l  J  n  K-*»  j  =  l 

and  the  result  follows  as  above. 

To  incorporate  B  <  III+SII,  one  makes  an  obvious  modification  in  the  above 

argument.  Each  FLPA  (S^)  is  chosen  to  contain  B  in  its  partition  (a^),  and 

1/ 

the  sums  are  taken  (for  J>)  only  over  those  j  such  that  1  +  0^  <  B. 

Similarly,  the  (p'j*)  and  (P^'n)  are  summed  only  over  tnose  values  <  B. 

For  unbounded  S,  since  n  if  fixed,  for  any  e  >  0  there  exists  M(e)  such 

^  tl  p  o 

that  when  M  >  M(e) ,  then  II  2  /  A  dE^x  -  Sxll  <  ellxll  .  for  all  x  in  Hw  . 

n=0  n- 1  n 

M  n 

That  is,  2  J  AdEL  converges  to  S  in  tne  strong  (pointwise)  operator 
n=0  n-1  A 

topology  on  2)(S);  however,  restricted  to  the  finite-dimensional  subspace  . 

n 

the  convergence  also  takes  place  in  the  norm  topology.  One  can  thus  apply  the 
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M  n 

preceding  to  the  bounded  operator  3^-2  f  AdE^.  and  then  let  M  -»  00  to 

n=0  n-1 

obtain  the  result  for  S.  □ 

Lemma  3  is  a  key  result  in  determining  the  capacity.  It  will  be  applied 

to  show  that  the  capacity  is  completely  determined  (for  a  given  set  of 

constraints)  by  (G^.  n  >  1),  where  Gn{A)  =  —  [#  eigenvalues  of  <  A], 

Several  results  that  demonstrate  the  importance  of  the  essential  spectrum 

CTess(S)  to  the  channel  capacity  will  now  be  obtained.  Let  V[S.(H^)]  be  the 

set  of  all  a  in  IR  such  that  for  any  K  and  any  a  >  0,  there  exists  n  >  K  such 

that  the  number  of  elements  in  the  sequence  (/3™)  satisfying  1/3™  —  nr  |  <  a  is 

>  K.  Note  the  importance  of  V[S,(H  )].  For  fixed  A.  lim  —  {#  eigenvalues  of 

n 

—  1  K~l 

S  <  Al  =  lim  —  2  [#  eigenvalues  of  S  in  fa., a.  ,)]  where  a.  <  a.  ,, 

n  J  n  .  &  n  L  i  i+l'J  l  l  +  l 

n  i=0 

0  <  i  <  K.  a^  =  -1,  and  a^  =  A.  For  a  <  A.  let  a^  =  (a-e,  a+e).  Then 

a  €  V[S.(Hn)]  is  a  necessary  condition  for  lim  ^  [#  eigenvalues  of  in 

(a-e.  a+e)]  >  0  for  every  a  >  0. 

Lemma  L .  Let  A  be  a  K-dimensional  subspace  of  H,  with  P ^  the  projection 

2 

operator  with  range(P^)  =  A.  If  (K+I)e  <  i,  then  IIP^z.ll  >  1  -  a  for  at 

most  K  elements  of  the  o.n.  set  (z,,z„ . z„  ,}. 

'12  K+l ’ 

1  2 

Proof .  Let  A  be  the  span  of  the  o.n.  elements  g . g...  Then  2\w  IIP.  z.  II  = 

1  K  l  =  1  A  i 

^=i<zi-%>2  -  4i  Iil!<ziigk>2  *  K-  if  "Vi"2  >  1  -  ‘  f°r 

i  <  K.  then  IIP.z..  .II2  <  K  -  K(l-e)  =  Ke  <  1  -  e  if  e  <  r^-p.  □ 

A  K+l  v  ’  ~  “K+l 

Pro2_^_J_.  Suppose  that  (u^,  n  >  1}  is  a  c.o.n.  set  for  H.  Let  (z^, 

2 

n  >  1}  be  an  o.n.  sequence  in  H  such  that  II (^S-a J }z^ll  -+  0.  Then 
-r  €  V[S.(Hn)]. 
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Remark.  The  existence  of  an  infinite  o.n.  sequence  (z^)  such  that 
2 

II (S — r I ) zn II  -»  0  is  a  necessary  and  sufficient  condition  for  tt  C  CTess(S)  [7]  . 
Thus.  Prop.  1  states  that  aess[S]  c  V[S,(Hn)]. 


Proof .  Suppose  that  there  exists  a  >  0  and  K  >  1  such  that 

dim[span{v^:  |/3^ — r|  <  e}]  <  K  for  all  n  >  1.  By  assumption,  and  using  the 

fact  that  the  monotone  family  of  projection  operators  (P^  )  converges  in  the 

n 

strong  topology  to  the  identity, 

0  =  lim  II(S-tI)z.II2  =  lim  lim  ll(S — r I ) Ptf  z.ll2 
i  in  n 

2 

>  lim  lim  IIP^  { S— -r I ) P^  zjl  .  so  that 

inn  n 

n  q  q 

0  =  lim  lim  2  ((5n.  -  ~t )  <z.,  v°>  . 
i  n  j=l  J  4  J 

Thus,  for  any  6  >  0  there  exists  1q(6)  such  that  for  all  i  >  i Q ( 5 ) . 

6  >  lim  2  (P1?  -  -r)2<z.  ,  vn>2. 
n  j=l  J  1  J 

Ordering  {v”,  i  <  n}  such  that  -  1 1  >  a  for  K+l  <  i  <  n,  we  have 
n  o 

6/a  >  lim  2  <z..  v.>  for  i  >  i_(6).  For  fixed  6,  choose  k. ,k0 . kv  . 

n  j=K+l  1  J  0  1  2  K+l 

such  that  k^  >  1q(6)  for  1  <  i  <  K+l.  For  any  a  >  0,  A  >  0,  there  exists 

2 

n^a.A)  such  that  n  >  n^a.  A)  implies  that  IIP^  z^  II  >  1  -  A  and  6/a  +  a  > 


2  <z.  ,  v.>  for  1  <  i  <  K+l.  For  such  an  n,  let  A  be  the  span  of 

j=K+l  ki  J  -  "  n 

(v" . vjj}.  Then  for  i  <  K+l,  IIPA  zk  II2  >  IIPW  z^  II2  -  ( 6/a  +  a)  > 

n  i  n  i 

1  -  A  -  ( 5/a  +  a).  Since  A,  5,  and  a  can  be  selected  so  that 
[K+1](A  +  6/a  +  a)  <  1,  one  obtains  a  contradiction  of  Lemma  4. 
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Prop.  1  gives  V[S,(H  )]  D  a  (S) ,  so  long  as  (u  ,  n  >  1}  is  complete  A 

n  g  s  s  ri 

partial  converse  is  given  by  the  following  result. 

Prop.  2.  Let  0 ^  and  0^  denote  the  smallest  and  Largest  points  in 
aess(S)-  Then  V[S.(Hr)]  c  [eL,en]. 

Proof .  It  will  first  be  shown  that  no  point  in  V[S,  (H  )]  can  exceed  9..  when 


S  is  bounded,  n  is  in  aess(S )  if  and  only  if  li(S-'rI )z^l 


U 

0  for  some  o.n. 


sequence  (z^) ,  and  this  requires  that  <Sz^,zn>  -*  nr .  In  order  that  nr  >  0^  be 
in  V[S,  (H  )].  it  is  necessary  that  for  any  e  >  0  and  M  <  00  there  exists  n  >  M 

such  that  I  ^  e  for  i(*0  =  1*2 . M.  This  requires  that 

|<Sz.,z.>  -  nr !  <  e  for  at  least  M  o.n.  elements  (z.)  in  H.  For  6  >  0.  let 

A(0y,5)  be  the  set  of  normalized  x  in  2)(S)  such  that  x  is  an  eigenvector  of 

for  some  n  >  1  and  the  corresponding  eigenvalue  is  >  0^  +  6.  Let  M  be  the 

maximal  number  of  o.n.  elements  in  A(0y,6).  <  00  for  every  6  >  0  implies 

that  V[S,  (Hn)]  contains  no  number  greater  than  0^. 

The  operator  I  -  Eg  +g/2  *s  con’Pact-  since  it  must  have  finite¬ 
dimensional  range  space.  Hence,  if  is  not  finite,  then  for  any  n  >  1  there 


exists  z  in  A(0„,6)  with  llz  11  =  1  and  11(1  -  E,,  ,„)z  I 

n  U  ’  n  v  0^+o/2'  n 

sup  <Sx,x>.  Then,  for  any  e  >  0, 

11x11=1 


<  — .  Let  t  = 
n 


T+e 


t  +  Gg  +  6  <  <(I+S)zn,  zn>  =  J  (l+X)dllExznl 


0J.+6/2  T+e 

S  ( l+A)dllEL  z  II  +  J  (l+AJdllE^z  II 

-1  n  0^+6/2  '  ’* 


V6/2  2  2 

$  ^  iwid^y *  (Iloilo  -  eV6/2)v 
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< 


Q..+6/2 

ux 

-1 


(l+AJdll^z  ll' 


+  (l+T+e.)/n 


<  (1+0^672)  +  (l+T+fc)/n. 


For  sufficiently  large  n,  this  is  a  contradiction. 

A  similar  approach  shows  that  every  number  in  V[S,  (H^)]  is  >  0  since 

E_  has  finite-dimensional  range  for  every  6  >  0.  0 

0l-6 

The  results  of  Prop.  1  and  Prop.  2  lead  naturally  to  several  reasonable 

hypotheses'  that  V[S,(H  )]  C  a  (S),  that  V[S,(H  )]  =  o  (S)  when  {u  , 

n  css  n  css  ri 

n  >  1}  is  complete,  and  that  lim  —  [#  eigenvalues  of  S  <  A]  is  independent  of 

n 

the  choice  of  the  c.o.n.s.  {u ,  n  £  1}.  Unfortunately,  all  three  of  these 
desirable  attributes  are  false,  in  general. 


Prop ,  3 . 

(1)  If  {u^,  n  >  1}  is  not  complete  for  H,  then  V[S,(H '  )]  fl  aess(S )  -  <t> 
is  possible. 

(2)  If  {u^,  n  >  1}  is  complete  for  H,  then: 

(a)  V[S,(H 'n)]  C  CTess(S)  is  not  always  true. 

(b)  Let  be  the  number  of  eigenvalues  (3™  of  such  that 

|P?‘x|  >  A  >  0  for  all  x  in  a  (S) .  Then  lim  —  Q.(S,H  )  can 

be  strictly  positive . 

_  1 

(c)  lim  —  [#  eigenvalues  of  S  >  AJ  may  not  be  independent  of  the 

n 

choice  of  {u  ,  n  >  1}. 

n  ~ 


Proof .  Counterexamples  will  be  constructed,  each  using  the  operator  S  defined 
as  follows.  Let  {v^.  n  >  1}  be  any  c.o.n.  set  in  H,  take  /3  >  a  >  0,  and  let 
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S  =  a  I  v  ®v  +  P  2  v  ®v  . 
n>l  n  n  n>2  n  n 


n  odd 


n  even 


To  prove  (1),  set 


u  = 
n 


V2n-1  *  v2n 

vS 


n  >  1 . 


Then  (Pw  SPW  )u.  =  p^lu.,  i  <  n.  n  >  1.  while  a  (S)  =  {a,p}. 

W  W  1  I  A  I  1  0SS 

n  n  V  V 


n  n 

To  prove  2(a),  choose  {u  ,  n  >  1}  by 


n 


_  Vn  4  Vn+(k+l)10k+1-klQk 

v5 


v  —v  1  1 c 

_  n  n-(k+l)10  +k!0 


2klOk  <  n  <  (k+l)10k+1  +  klOk 


(k+1 ) 10k+1  +  klOk  <  n  <  2(k+l)10k+1 


for  k  =  0,1,2,...  (P^)  then  has  the  following  composition: 


n  =  (2k)10  :  (£F)  =  (a,p.  each  of  multiplicity  klO  ) 

n  =  (k+l)10k+*  +  klOk:  (/3*?)  =  (a,/3,  each  of  multiplicity  klOk) 

U  each  of  multiplicity  (k+l)10k  '  -  kl0‘ 

For  2(b),  note  that  if  atP  <  X  <  li,  then  the  above  choice  of  (u  ,  n  >  1} 

£  n 


gives  lim  —  [#  eigenvalues  of  S  <  X]  =  lim  £k--^.-Q 
n  n  n  k-*»  (k+1) 10 


k+1 


10 


r-  =  77  .  Fur  ther  , 
k  1 1 


k-*»  (k+l)10k+1+kl0 
k+1  k 

r__  ,  x  ,P  T7-  l  n  m  \  (k+1)  10  -klO  9 

for  A  >  (h-  -  o),  —  Qi(S,H  )  =  lim  -1 - L — r— - p  =  7-7-  . 

2  2  n  Av  n'  ,  _  M.nmk+lMink  11 

n  k-*»  (k+1)  10  +kl0 

Finally,  2(c)  is  shown  by  using  {u  .  n  >  1}  defined  by  u  =  v  ,  n  >  1, 

1  n  ~  1  n  n 

which  yields  (j3^)  containing  only  a  and  /3 ,  each  of  multiplicity  n/2  (n  even) 
or  a  with  multiplicity  2£i.t  p  with  multiplicity  (n  odd).  □ 
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The  main  result  can  now  be  obtained. 


Theorem. 


_  B  rB  +li 

Urn  A  f1  log  dFjX)  <  C 'y(P) 

n-*°  -1 

_ b  rB  +1i 

<  UmA  f1  log  -  fc  dFn( X)  +  £  1%  {N) 
n-*°  - 1 


where  B  (n  £  1)  is  defined  by 


P  =  f1  (Bn-\)dFn(X) 


F  (X)  =  —  [#  eigenvalues  of  <  X J . 

(2)  If  lim  —  [#  eigenvalues  of  S  <  XJ  exists  for  all  X  in  IR ,  then 

n-*° 

'A  log[^-]dF(X)  i  C J(P)  <  K  X?1  log/|±j.JdF(X)  +  ITm  £  H^(N) 

n 

where  F  is  a  distribution  function  defined  by 

F(X)  =  lim  l-[#  eigenvalues  of  S  <  XJ  =  lim  F  (X), 
n-f”  n-*°  n 

and  the  constant  B  is  defined  by 

P  =  J^fB-XJdFfX). 

(3)  If  lim  ^  H^(N)  =  0.  then  C^(P)  =  0  if  and  only  if  lim  ^  [U 

n  n-*° 

eigenvalues  of  <  Xj  =  0  for  all  X  in  IR.  This  requires  that  S  be 
unbounded  and  always  occurs  if  I1®0  is  the  only  limit  point  of  o(S) . 

Proof.  It  is  sufficient  to  evaluate  lim  —  C^(nP).  Applying  Lemma  3  to  the 

n 

expressions  contained  in  Lemma  1, 


Info. Cap.  of  DLC  -  9/5/89  -  15 


B  r  B  +1t  B  B  +1 

“  £  Mt£tK<x>  *  *  {"  ^[-xrr]^^) *  z  >&<*> 

1  B 

where  G^fX)  =  —  [#  eigenvalues  of  Sn  <  X]  and  P  =  Jn  [B^-XjdF^X)  ,  n  >  1. 

Note  that  since  the  integrands  appearing  in  the  statement  of  (1)  are 

equal  to  zero  when  evaluated  at  the  upper  limit,  and  are  continuous,  the  left- 

continuous  function  Gn(X)  =  ^  [#  eigenvalues  of  Sn  <  X]  can  be  replaced  by 

G  (X+)  =  F  (X).  The  same  comment  holds  for  the  remainder  of  the  theorem, 
n  n 

To  prove  (2).  we  can  suppose  that  lim  ^  H^^N)  =  0.  Then,  let  (n.)  be 

n  J 

any  subsequence  of  the  integers  such  that 

_  l  -  B(V  rB(n  )*U 

lim  i  cJ(nP)  =  lim  'A  f  log  — — dFn(X) 
n  j-*»  -1  L  nj 

Let  B  be  any  finite  limit  point  of  (B(nj),  j  £  1)  (assuming  that  at  least  one 
exists).  Then,  restricting  attention  to  (n  )  such  that  B(n  )  -♦  B. 

J  J 

„  B(nj}  rB(nJ+h 


«  J  pin  ;+i-i 

2cw(p)  =  lim  S  l°g[-xir-JdFn  (X) 

J  1  J 

B  rB(n  )+h  B  rB(n  )+l-i 

=  lim  S  log  — -  dF  (X)  -  f  log  — -  dF  (X) 

J— l-l  i  X+1  ‘  "j  B(n . )  L  X+1  J  "j  J 


B  rB(n  )+l- 

lim  f  log  — 5-^ - 

J—  B(n  j )  M1  ■ 


pinli+1l  B(n  )+l 

X+1  dFn  (X)  $  li-  log  -riH  (Fn  (B)  ~  Fn  (B(n  )) 

j  j-*®  j  j  J 


i  lim  log 


B(n  )+l 


lh°- 


We  thus  obtain 
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«,  B  rB(n  )+li 

^(P)  =  lim  log[  jdFn  (A) 

B  B(n  )+l  B+1 

=  lim  *  los[— b^"]  +  log[5^ 

jHOO  -1 

|B  rB(n  )+li  I  |  rB(n  )+l 


B(n . )+l  i 

-  b+t  ^ +  1o^  dFn.<X> 


°  ji-m  r«in  .j-t-i-i 

and  since  /  log  — =“■: -  dF  (X)  <  log  — s*-: ,  one  has 

,  I  B+ 1  n .  ti+  i 

1  .1 


(^(P)  =  lim  *  s  log[^]dFn  (X) 
j*°  -1  j 


and,  similarly. 


P  =  lim  X  [B  -  X]dF  (X). 

J-*»  -1  nj 

Assuming  that  F  (X)  -»  F(X)  for  all  X  in  IR .  this  gives  (2)  of  the  Theorem 

"j 

whenever  (B(n^.),  j  >  1}  has  a  finite  limit  point.  Moreover,  in  this  case  it 

can  be  seen  that  every  maximizing  sequence  must  have  B  as  a  limit  point. 

To  complete  the  proof  of  (2)  and  simultaneously  prove  (3),  suppose  that 

(n.)  is  as  above  and  that  no  maximizing  sequence  (B(n.),  j  >  1)  has  a  finite 
J  J 

B(n  ) 

limit  point.  Since  P  =  J  [B(nj)  -  X]dFn  (X)  for  all  j  >  1,  and  B(n^)  -*  °°, 

J 

it  follows  that  lim  F  (X)  =  F(X)  =  0  for  every  finite  X  and  every  maximizing 

j-*0  nj 

00 

sequence  (nj)-  Thus,  in  this  case  C^(P)  has  the  same  value  as  if  +00  were  the 
only  limit  point  of  cr(S);  assume  WLOG  that  this  is  true.  For  a  fixed  n,  the 

~  o 

channel  in  H  is  given  by  X  +  P  N  =  Y,  where  Uv[H  ]  =  1  and  E  llxIL,  <  nP.  A 
n  &  J  n  n(L  nJ  p^  W,n  ~ 

given  x  in  has  the  representation  x  =  ^v  for  some  unique  v  in  H  ,  and 

llxllw  ^  =  llvll.  Vn  is  isometric  on  range(R^Pn) .  The  channel  is  thus 

2 

equivalent  to  the  1L,  channel  Y  =  X  +  V  P  N,  wi  th  X  a.  s .  in  H,.,  ,  E  llxll,.,  = 

f>n  n  n  W,n  p^,  W 

E^IIR^xll^  <  nP.  For  this  n-dimensional  channel,  C^(nP)  <  C^(nP  ,  max) .  where 
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C^(nP,  max)  is  the  information  capacity  of  the  infinite-dimensional  Gaussian 

channel  when  the  noise  has  covariance  operator  and  the  transmitted  signal 

measure  must  satisfy:  dim[supp(p^)]  £  n.  p^[range(R^) ]  =  1,  and 
2 

E  llxllw  £  nP.  From  Theorem  2  of  [2] 

MX  » 


K(n) 

c£(nP,  max)  =  'A  2  log 
i=l 


C(nl' 


'i  J 


where  X^  £  £  X^  <  ...  are  the  eigenvalues  of  I+S  (since  S  has  a  finite  set 

of  limit  points,  it  has  pure  point  spectrum); 

K(n) 


and 


Thus , 


P  =  i  2  [C(n)  -  X  ] 

1=1 


K(n)  =  sup{i:  X^  <  C(n)}. 


t  pn.  p  w  1  K(n)[C(n)-Xll 

-  CJ(„P.  max)  4  ^  ^  [-jq— ] 


1  K(n)  p 

and  since  X^  ^  »  and  ^  2  [C(n)-X^]  =  this  gives 

Tim  ^  C^(nP)  ^  THiT  ^  cJ(nP,  max)  =  0. 


Thus,  for  a  maximizing  subsequence  (n  ),  either  (B(n  ))  has  a  finite 

J  J 

limit  point  B,  or  else  B(n  )  -»  In  the  first  case,  (1)  and  (2)  (when 

J 

00 

applicable)  of  the  Theorem  give  the  capacity.  In  the  second  case,  C^(P)  =  0. 

It  is  clear  that  either  every  maximizing  sequence  (n.)  has  (B(n.))  with  a 

J  J 

finite  limit  point,  which  has  the  unique  value  B,  or  else  (B(n^))  has  no 
finite  limit  point  for  any  maximizing  sequence  (nj)-  D 
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Remark .  In  part  (2),  the  same  result  holds  so  long  as  F(X)  =  lim  F  (X)  exists 

n-*° 

for  all  X  <  B.  with  B  defined  as  in  (2). 


The  following  result  is  a  slight  refinement  of  part  (1)  of  the  Theorem; 
the  direct  proof  is  omitted. 

Corollaru  1.  Suppose  that  lim  ~  C^(nP)  >  0  and  that  lim  k 

n  n 

and  let  8  be  the  largest  number  such  that  P  2  lim  /^[B^  -  X]dF^(X) .  Then 


_  ,  _  B  rB  +1 

lim  cJ(nP)  =  It*  /  log1  n 
n  -1 


n 


X+i 


dfn(X). 


where  (B^)  and  (F  )  are  defined  as  in  Theorem. 


Corollaru  2.  When  lim  —  »  0,  then  bounds  on 

n  UN 
n 

K  log(l  *  P/X^)  £  CJ(P)  £  X  log(l  +  P/X^} 

where  X  .  Is  the  smallest  limit  point  In  the  spectrum  of  S,  and  X  is 
min  max 

the  largest.  Moreover,  these  bounds  can  be  attained  by  proper  choice  of 

(Hf  ). 

n 

Proof .  The  bounds  follow  from  Prop.  2.  The  upper  bound  is  attained  if  {u., 

i  2  1}  is  chosen  so  that  II (S — r J )u. .  II  ->  X  .  ;  such  an  o.n.  set  exists,  since 

l  min 

X  .  is  in  a  (S) .  The  lower  bound  is  attained  in  a  similar  manner.  □ 
min  ess 


C^(P)  are  given  by 


Applications 

A  number  of  applications  of  the  main  result  are  given  in  [3].  These 
include  new  results  for  nonstationary  discrete-time  channels  and  for  both 
stationary  and  non-s tat ionary  continuous-time  channels  when  the  time  of 
transmission  is  fixed. 


Info. Cap.  of  DLC  -  9/15/89  -  19 


Extensions 


For  the  classical  continuous-time  channel,  the  time  of  transmission  is 

permitted  to  increase  in  order  to  define  average  information  capacity.  The 

dimensionality  n  used  here  is  replaced  by  the  transmission  time  T,  and  the 

2 

subspace  H  is  replaced  by  Lo[0,T].  The  constraint  E  llxIL  i  nP  is  replaced 
n  n  w ,  n 

he 

2  T  Vi  2 

by  the  constraint  E  ^,11x11^  ^  <  TP,  where  ji^[range(R^  ^)]  =  1,  ^  = 

IIR^  -jjcILp  and  11*11^.  is  the  l^CO.T]  norm.  is  a  bounded  linear  operator  in 

LgCO.T]  defined  by  a  covariance  function  r^.  with  r^(t,s)  defined  for  all  s. 
t  1  0. 

The  method  used  here  can  also  be  used  to  analyze  capacity  of  such 
channel®  Those  results  will  be  given  elsewhere. 
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